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Abstract 



Let rriij be the mean first passage time from state i to state j in an n-state ergodic 
homogeneous Markov chain with transition matrix T. Let G be the weighted digraph 
without loops whose vertex set coincides with the set of states of the Markov chain 
and arc weights are equal to the corresponding transition probabilities. We show that 




fij/qj, if i^j, 



where fij is the total weight of 2-tree spanning converging forests in G that have one 
tree containing i and the other tree converging to j, qj is the total weight of spanning 
trees converging to j in G, and q~j = qj/Yl^ilk- The result is illustrated by an 
example. 
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1 Introduction 

Let T = (tij) G M. n ' n be the transition matrix of an n-state ergodic homogeneous Markov 
chain with states 1, . . . , n. Then T is an irreducible stochastic matrix. The mean first passage 
time from state i to state j is defined as follows: 





k=l 
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where 

Fij = min{p > 1 : X p = j | X = i}. (2) 
By [TJ Theorem 3.3] the matrix M = (mij) nxn has the following representation: 

M = (I-L# + JL*)U~\ (3) 

where 

L = I-T, (4) 

is the group inverse of L, L* g is the diagonal matrix obtained by setting all off-diagonal 
entries of L* to zero, II = diag(7Tx, . . . , 7T n ), and (7r 1; . . . , 7r n ) = it is the normalized left 
Perron vector of T, i.e., the row vector in M. n satisfying 

7rT = 7r and ||7r||i = 1. (5) 

In an entrywise form, (j3J) reads as follows (see, e.g., [2]): 

_ jnj 1 , if i =j, 

In the following section, we use this formula to derive a graph 
the mean first passage times. 



(6) 

-theoretic interpretation of 



2 A forest expression for the mean first passage times 

Let us say that a weighted digraph G corresponds to the Markov chain with transition ma- 
trix T if the Laplacian matrix L of G satisfies (j3J). Let the vertex set of G be {1, 2, . . . , n}. 
Then arc belongs to the arc set of G whenever [i ^ j and Uj ^ 0]; the weight of this 

clXC IS t"ij . 

We will use the following graph-theoretic notation. A digraph is weakly connected if 
the corresponding undirected graph is connected. A weak component of a digraph G is any 
maximal weakly connected subdigraph of G. An in-forest of G is a spanning subgraph of G all 
of whose weak components are converging trees (also called in-arborescences) . A converging 
tree is a weakly connected digraph in which one vertex, called the root, has outdegree zero 
and the remaining vertices have outdegree one. An in-forest is said to converge to the roots 
of its converging trees. An in- forest F of a digraph G is called a maximum in-forest of G if G 
has no in-forest with a greater number of arcs than in F. The in-forest dimension of a digraph 
G is the number of weak components in any maximum in-forest. Obviously, every maximum 
in-forest of G has n — d arcs, where d is the in-forest dimension of G. A submaximum in-forest 
of G is an in-forest of G that has d + l weak components; consequently, it has n — d — 1 arcs. 

The weight of a weighted digraph is the product of its arc weights; the weight of any 
digraph that has no arcs is 1. The weight of a set of digraphs is the sum of the weights of 
its members. 
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By [21 (iii) of Proposition 15], for any weighted digraph G and its Laplacian matrix L, 

L* = a~ l _ d (Q n - d -l ~ ^^Qn- d ) , (7) 

V °n-d J 

where <Jk is the total weight of in-forests with k arcs (so that 0" n -<2 and o n -d~i are the total 
weights of maximum and submaximum forests of G, respectively), Qk is the matrix whose 
entry = 1, ...,n) equals the total weight of in-forests that have k arcs and i 
belonging to the tree that converges to j. 

Let us combine flB]) and (J7|) to obtain a forest representation of the mean first passage 
times. First, observe that since the Markov chain under consideration is ergodic, the cor- 
responding digraph G has spanning converging trees. Thus, the forest dimension of G is 1. 
Consequently, for every i,j = 1, . . . , n, each maximum in-forest converging to j has i in the 
same (unique!) tree. Therefore, the and entries of Q n -d = Q n -i are the same. 
The difference 

fu Q% 1 ■ <ih 2 (8) 

between the (j, j) and entries of Q n _d-i = Qn-2 — (<2i}~ 2 ) equals the weight of the set 
of 2-tree in-forests of G that converge to j and have i and j in separate trees. We have that 
substituting (J7|) in (jfJJ) yields = /jj/(cr n -i 7Tj) if i 7^ j- Finally, we know (a more general 
result is the Markov chain tree theorem [HE]) that 7Tj = qj/a n -i, where qj is the total weight 
of trees converging to j in G, so that Y^k=x ( lk — °n-i- This provides = fij/qj if i ^ j 
and rriij = cr n -i/Qj if i = j- We obtain the following theorem. 

Theorem 1 Let T G M n,ri be the transition matrix of an n-state ergodic homogeneous 
Markov chain with states l,...,n. Let G be the weighted digraph without loops whose ver- 
tices are 1, . . . , n and arc weights are equal to the corresponding transition probabilities in T. 
Then the mean first passage times from state i to state j can be represented as follows: 

[1/qj, if %=3, 

where is the total weight of 2-tree in-forests of G that have one tree containing i and the 
other tree converging to j, qj is the total weight of spanning trees converging to j in G, and 

Remark 1. If one removes the subsidiary requirement p > 1 from the definition of mean 
first passage time (Eq. (T5])), then rrijj = for every j and my = fij/Qj, h j = 1> ■ ■ • , n , since 
fjj = for ever y J- 

Remark 2. fij and qj can be calculated by elementary matrix algebra. Namely, this can 
be done by means of the following recurrence procedure [H Proposition 4]. For k = 0,1, . . . , 

ak+1 = TTT' (10) 

Qk+i = —LQk + Vk+iI, (11) 

where Qo = I. 
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3 An example 



In this section, we illustrate Theorem [T] and Remark 2 by an example. Let 



T 
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First, let us obtain the matrix M of the mean first passage times by the direct use of ([3 
Finding 

7T = (0.08,0.4,0.2,0.32) 



and 



L* 



0.7408 1.704 -0.448 -1.9968 

-0.1792 2.104 -0.248 -1.6768 

0.2208 -0.896 0.752 -0.0768 

-0.0992 -2.496 -0.048 2.6432 



(12) 



(13) 



enables one to compute 



M 



12.5 


1 


15 


14.5 


11.5 


2.5 


5 


13.5 


6.5 


7.5 


5 


8.5 


10.5 


11.5 


4 


3.125 



(14) 



Mention here that one of the methods to calculate is by applying 

L# = (L + J)- 1 -J, 

where 



J 



'1,1, 



lfvr. 



(see, e.g., [31 (i) of Proposition 15]). 

In what follows, we obtain M by means of Theorem [TJ 

The digraph G corresponding to the Markov chain is shown in Fig. [IJ 

The converging trees of G are shown in Fig. [21 where the roots are given in a boldface font. 
By the definition of qi given in Theorem (U 



q = (gi,«j,?3,g 4 ) = (0.02,0.1,0.05,0.08). 



(15) 
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Figure 1: The digraph corresponding to the Markov chain. 




Figure 2: The converging trees Ti,T 2 ,T 3 , and T 4 of G. 



Since J2t=i 9* = 0-25, it follows from f|T5|) that 
5 = (91,92,93,54) 



Sfc=i 5i 



(0.08,0.4,0.2,0.32). 



(16) 



This vector coincides with 7r, the normalized left Perron vector of T. 

The 2-tree in-forests of G are shown in Fig. [3X where the roots are given in a boldface 
font. 

In Theorem [T], /y is defined as the total weight of 2-tree in-forests of G that have one 
tree containing % and the other tree converging to j. Therefore, 



(A 



^({^3}) M{F 2 ,F 5 }) w({F 1 ,F 4 ,F 6l F 7 ,F 8 }) 

w({F 2 ,F 3 ,F 7 }) M{F 5 }) ^({Fx,^,^,^}) 

w({F 2 ,F 7 }) w({F 3 ,F 5 ,F 8 }) ^({Fi,^,^}) 

W ({F 1; F 2 ,F 7 }) w({F 3 ,F A ,F 5l F 8 }) w({F 6 }) 

0.1 0.75 1.16 

0.23 0.25 1.08 

0.13 0.75 0.68 

0.21 1.15 0.2 



(17) 
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w(F 5 ) = 0.25 





w(F 6 ) = 0.2 w(F 7 ) = 0.08 w(F 8 ) = 0.4 



Figure 3: The 2-tree in- forests Fi, . . . , F s of G. 
Together with (fToT) - (|T7|) Theorem [1] provides the matrix of the mean first passage times: 
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M 



In accordance with Remark 2, to apply Theorem [TJ it is not necessary to generate the 
converging trees and 2-tree in-forests of G as we have done before. Instead, and qj can 
be computed by means of the recurrence procedure (ITOl - tfTTl) . Starting with Q = I, for our 
example, we have 



0"! 



tr(LQo) 



2.25, 



Qi 
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-LQ + ail 
tr(LQ0 
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1.56, 



6 



0-3 



Qi 



-LQi + a 2 I 
tr(LQ 2 ) 



0.25, 



LQ 2 + a 3 I 



0.31 1.05 0.2 

0.08 1.15 0.25 0.08 

0.18 0.4 0.5 0.48 

0.1 0.3 1.16 



0.02 0.1 0.05 0.08 

0.02 0.1 0.05 0.08 

0.02 0.1 0.05 0.08 

0.02 0.1 0.05 0.08 



(19) 



Denoting Q 2 = (qfynxn, by (ED we have /, 



ij 



q 2 . 



Qtj, hJ 



(./; 



13 1 



0.1 0.75 1.16 

0.23 0.25 1.08 

0.13 0.75 0.68 

0.21 1.15 0.2 



(20) 



, n, and ( fill provides 



(21) 



which coincides with ( !T7l) ; Eq. (|20l yields q = (0.02,0.1,0.05,0.08), which coincides with 
( TT5l) . Thus, Theorem [1] provides the matrix ( JTBl of the mean first passage times again. 
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